Papers with length-based models
FLARE: Fine-Grained Length-Aware Routing for Resource-Efficient Heterogeneous LLM Serving (2026.acl-long)
Copied to clipboard
| Challenge: | Existing routers lack fine-grained resource awareness across deployment settings, which degrades efficiency metrics in real-world serving. |
| Approach: | They propose a length-centric, resource-aware multi-LLM routing framework that uses length-based models to estimate per-query latency and cost. |
| Outcome: | Experiments show that FLARE reduces latency and cost by up to 68% and 75% while maintaining competitive accuracy. |